ilastik: interactive machine learning for (bio)image analysis

We present ilastik, an easy-to-use interactive tool that brings machine-learning-based (bio)image analysis to end users without substantial computational expertise. It contains pre-defined workflows for image segmentation, object classification, counting and tracking. Users adapt the workflows to the problem at hand by interactively providing sparse training annotations for a nonlinear classifier. ilastik can process data in up to five dimensions (3D, time and number of channels). Its computational back end runs operations on-demand wherever possible, allowing for interactive prediction on data larger than RAM. Once the classifiers are trained, ilastik workflows can be applied to new data from the command line without further user interaction. We describe all ilastik workflows in detail, including three case studies and a discussion on the expected performance

Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images

We describe a protocol for fully automated detection and segmentation of asymmetric, presumed excitatory, synapses in serial electron microscopy images of the adult mammalian cerebral cortex, taken with the focused ion beam, scanning electron microscope (FIB/SEM). The procedure is based on interactive machine learning and only requires a few labeled synapses for training. The statistical learning is performed on geometrical features of 3D neighborhoods of each voxel and can fully exploit the high z-resolution of the data. On a quantitative validation dataset of 111 synapses in 409 images of 1948×1342 pixels with manual annotations by three independent experts the error rate of the algorithm was found to be comparable to that of the experts (0.92 recall at 0.89 precision). Our software offers a convenient interface for labeling the training data and the possibility to visualize and proofread the results in 3D. The source code, the test dataset and the ground truth annotation are freely available on the website http://www.ilastik.org/synapse-detection

Correlative In Vivo 2 Photon and Focused Ion Beam Scanning Electron Microscopy of Cortical Neurons

Correlating in vivo imaging of neurons and their synaptic connections with electron microscopy combines dynamic and ultrastructural information. Here we describe a semi-automated technique whereby volumes of brain tissue containing axons and dendrites, previously studied in vivo, are subsequently imaged in three dimensions with focused ion beam scanning electron microcopy. These neurites are then identified and reconstructed automatically from the image series using the latest segmentation algorithms. The fast and reliable imaging and reconstruction technique avoids any specific labeling to identify the features of interest in the electron microscope, and optimises their preservation and staining for 3D analysis

Manual reconstruction of a bouton its synaptic partner, and all membrane organelles contained.

<p><b>A</b>, <b>B</b>, The FIBSEM image series can be used to segment <i>in vivo</i> imaged structures (<b>A</b>, inset) including all their organelles: axonal bouton – yellow, mitochondria – green, synapse – red, synaptic vesicles – gold, endoplasmic reticulum – blue, dendritic spine – pink, dendritic endoplasmic reticulum – orange. The reconstruction is made from an image volume (6.4 µm×8.0 µm×5.0 µm) (<b>B</b>, inset left) that also includes the synaptically coupled dendritic spine (<b>B</b>, inset right). Scale bar in <b>A</b> is 1 µm, and in <b>A</b> (inset) is 5 µm.</p

<i>In vivo</i> imaging, laser branding and tissue preparation.

<p><b>A</b>, Cortical surface showing the vasculature on the surface of the brain. Dotted lines indicate the blood vessels that can also be seen as dark shadows in the 2PLSM (<b>B</b>), with the white square indicating the region imaged at higher magnification (inset). After fixation and sectioning this region (<b>C</b>) was then laser branded, and reimaged using 2PLSM. These branding marks were visible (<b>D</b>) in the resin block (indicated with white arrow heads) without any further enhancement. Their position is also highlighted with laser etching on the surface (black arrows) that can be seen in the FIBSEM (<b>E</b>). This indicates the region to be imaged (<b>F</b>) so that imaging and milling will capture the branded region (white arrow heads). Scale bar in <b>A</b> and <b>B</b> is 100 µm, and 10 µm in (<b>C</b>–<b>F</b>).</p

Interactive segmentation of neurites.

<p><b>A</b>, The ilastik software allows users to select regions for segmentation (yellow box), and compare the generated 3D model (<b>B</b>) with the <i>in vivo</i> image (<b>C</b>) of the neurite of interest. A GFP dendrite from a layer 5 pyramidal neuron is shown in <b>C</b>. The method can also be used with other fluorescent markers, such as tdTomato (<b>D</b>), here expressed in GABAergic axons and dendrites in a different animal. When the density of neurites (<b>E</b>) is high the neurites can still be distinguished. Panel <b>E</b>, shows the reconstructed axons (red) and dendrites (grey) that are shown in the in vivo image in <b>D</b>. Scale bar in <b>C</b> and <b>D</b> is 5 µm.</p